Upgraded metadata lookup for backwards compatibility, added option to input metadata manually#179
Upgraded metadata lookup for backwards compatibility, added option to input metadata manually#179
Conversation
…written out during a scan
…here epoch defined.
…rched in Tiled for backwards compatibility
pbeaucage
left a comment
There was a problem hiding this comment.
This is actually a pretty minor change to the data structure that maps metadata fields to PyHyper words, made far more complicated by changing a bunch of variable names and re-inventing basic function calls. Before merging, we should
- revert the changes to variable names from lowercase/underscore to camel case, making them match the rest of the class. This made this quite difficult to review and I'd like to look again after this is fixed to ensure we didn't lose anything in the diff.
- add a test that covers the new metadata names
- ideally, add a documentation page on this variable name mechanism. I'd accept making an issue for this.
…ade during 20250308 beam time.
…d values for now.
Co-authored-by: Peter Beaucage <peter.beaucage@nist.gov>
pbeaucage
left a comment
There was a problem hiding this comment.
I believe this is now good to go, assuming you are OK with the workaround for manual metadata documented in the issue.
Also, can you please add some tests for new data before we merge?
Yes, your changes look good to me. I just added some tests. I noticed before adding my tests that some of the tests had failed. Possibly from the recent commits, but I hadn't kept a close eye. |
What do you mean regarding the third bullet? Is that for explaining PyHyperScattering variable names or beamline variable names? |
|
Tests failing: Weird, transient Tiled errors. I was looking at that. Hopefully they don't recur. You can click thru to get the run log. Variable names: a list of the standard PyHyper words and what they mean; Just curious: do those test load calls work without hinting the dimensions? I'm especially curious for the |
What do you mean by the first sentence? Is the "consequence" referring to some operation and the "flow control block" things like |
|
Blegh, tests are working, but the Tiled server is just dropping connections. I've messaged on NSLS2 slack. |
The energy scans (scan IDs 93065 and 91175) and the spiral (scan ID 92770) worked. Time scan (scan ID 92849) did not work. Error: |
So yes, while it is technically valid Python syntax to write, for instance: (or yes, any ofter flow control statement like One way to think about this is that Python is a language where whitespace and layout are meaningful, compared with say C. So following expected patterns of whitespace and flow are helpful. |
I'm testing more recent scans now and ran into a situation where scan ID 93983 loads fine with Error: |
|
OK, that sounds like a separate issue. Not sure what would be causing it offhand. Feel free to make new thread to look into. |
|
Tests pass! merging |
Sounds good. Created issue #193. |

Addresses issue #178.
mdLookupdictionary was updated so that beamline metaddata key names can be entered as a list instead of single value (or two values in the case ofsecondary_lookup table). Sets up infrastructure for backwards compatibility with respect to key names used historically at the beamline.secondary_lookupdictionary was removed, as all historical key names are consolidated intomdLookup. These changes were propagated to the code where Tiled is searched for metadata keys as well as in the construction of reverse lookup table (reverse_lut) inloadRun. These changes were tested for scan IDs 92849, 93065, and 91175, which include scans from before and after the beamline codebase upgrades in January 2025.Also tested the following:
Additionally,
mdManualinput was added intoloadRunfunction so that metadata values can be entered manually in case they were not originally written out with the scan. My understanding is that the currentcoordsinput cannot take more qualitative single-value entries (e.g.,{"sample_notes": "details"}) that are meant to be stored inattrs. If it makes sense, I could think of how to consolidate these two inputs.